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ABSTRACT 

This  document  introduces  a  series  of  documents  which  focus  on  the  conformance  test  specifications  for  the  various 
ISDN  Protocols.  An  overview  of  the  current  status  of  ISDN  Conformance  Testing,  and  the  issues  involved  are  also 
provided. 
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NOTICE  OF  DISCLAIMER 


This  document  contains  NIUF  conformance  tests  as  agreed  among  participating  expert  technical 
personnel  to  the  texts  of  isdn  standards,  conhgurations  and  descriptions  that  are  intended  to 
promote  interoperability  and  efficiency.  these  conformance  tests  were  developed  and  approved  by 
organizations  participating  in  the  north  american  isdn  users'  forum  (niuf)  meetings.  neither  the 
National  Institute  Of  Standards  And  Technology  (NIST)  nor  any  of  the  participants  in  the  NIUF  make 

ANY  representation  OR  WARRANTY,  EXPRESS  OR  IMPLIED  WITH  RESPECT  TO  THE  SUFFICIENCY,  ACCURACY,  OR  USE 
OF  ANY  INFORMATION  OR  OPINION  CONTAINED  HEREIN.  THE  USE  OF  THIS  INFORMATION  OR  OPINION  IS  AT  THE  RISK 
OF  THE  USER.  UNDER  NO  CIRCUMSTANCES  SHALL  NIST,  OR  ANY  PARTICIPANT  IN  THE  NIUF  BE  LIABLE  FOR  ANY 
DAMAGE  OR  INJURY  INCURRED  BY  ANY  PERSON  ARISING  OUT  OF  THE  SUFFICIENCY,  ACCURACY,  OR  USE  OF  ANY 
INFORMATION  OR  OPINION  CONTAINED  HEREIN. 


NIST  does  not  recommend  or  endorse  products  and  nothing  contained  herein  is  intended  as  a  recommendation  or 
endorsement  of  any  product. 
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1  INTRODUCTION 


Advances  in  communication  networks  have  increased  the  complexity  of  communications  protocols  and 
interfaces  significantly,  and  the  correct  interpretation  of  standards  has  become  a  critical  issue  in  ensuring 
interoperability  of  equipment  from  different  vendors.  The  challenge  to  the  telephony  industry  in  providing 
high  quality  Integrated  Services  Digital  Network  (ISDN)  service  and  interoperable  ISDN  products  is 
unprecedented,  as  the  ISDN  protocols  and  interfaces  are  very  complex  and  more  open  in  terms  of  how 
services  are  to  be  provided.  Conformance  testing  is  a  fundamental  task  in  meeting  this  challenge. 


This  document  is  one  of  a  group  of  National  Institute  of  Standards  and  Technology  (NIST)  Special 
Publications  that  will  be  issued  on  Integrated  Services  Digital  Network  Conformance  Testing.  The  purpose 
of  this  group  of  Special  Publications  is  to  publish  the  set  of  conformance  test  suites  which  were  agreed  upon 
in  the  North  American  ISDN  Users'  Forum  (NIUF).  These  documents  will  be  referenced  by  the  Federal 
Information  Processing  Standard  (FIPS)  which  specifies  the  testing  requirements  which  ISDN  equipment 
must  meet  to  comply  with  ISDN  protocol  standards  and  agreements  reached  in  the  NIUF. 


This  pubUcation,  which  is  an  introduction  to  the  group,  provides  the  general  scope  of  ISDN  conformance 
testing.  Section  3  provides  an  overview  of  the  abstract  test  suites  in  the  group,  including  the  test 
configurations  of  each  suite.  Section  4  ouUines  conformance  testing  standard  guidelines  and  describes 
conformance  testing  development.  In  addition,  sections  5  and  6  provide  information  on  the  use  of 
proformas  in  testing,  and  conformance  and  interoperability  testing. 


2  SCOPE  OF  ISDN  CONFORMANCE  TESTING 


Conformance  testing  is  the  process  of  determining  whether  an  implementation  complies  with  the 
specifications  of  the  standard  and  exhibits  the  behaviors  mandated  by  the  standard  [22].  There  are  two 
classes  of  testing  on  an  ISDN  network-user  interface:  testing  from  the  direction  of  the  network  and  testing 
from  the  direction  of  the  user.  Figure  1  shows  the  model  of  functional  groups  and  reference  points  across 
a  network-user  interface  [5]. 
The  functional  groups  include 
the  Exchange  Termination  (ET) 
group,  the  Network  Termination 
(NTl  and  NT2)  group.  Terminal 
Equipment  of  type  1  (TEl),  and 
the  Terminal  Adaptor  (TA).  The 
TE2  functional  group  represents 
the  existing  non-ISDN  data 
terminals. 


U 


TEl 


JnT2 


TE2 


TA 


NTl 


ET 


Reference  Points: 
R,  S,  T,  U 


Figure  1.  ISDN  Interface  Functional  Groups  and  Reference  Points. 


The  functional  grouping  shown 
in  figure  1,  represents  only  an 
abstract    model    for  easier 

reference  by  protocol  standards.  This  document  refers  to  equipment  to  the  left  of  the  U  reference  point  as 
the  User  Side  equipment.  An  actual  implementation  of  a  user  side  equipment  may  perform  more  than  one 
of  these  functions,  for  example,  it  may  perform  only  the  TEl  or  TA  function,  the  TEl  or  TA  and  NT2 
combined,  the  NTl  plus  NT2,  or  other  combinations.  A  TEl  or  TA  may  be  designed  to  connect  directly 
to  an  NTl,  in  which  case  the  S  and  T  reference  points  coincide.  When  the  TE  and  NTl  function  are 
combined,  only  the  U  reference  point  is  available. 


There  are  several  basic  test  configurations  of  the  System  Under  Test  (SUT)  and  Tester  for  conformance 
testing,  given  table  2.1,  below.  The  symbol  "TE"  in  the  table  2.1  rs  to  either  a  TEl  or  TA,  and  S/T  refers 
to  the  case  where  S  and  T  coincide.  For  testing  from  the  direction  of  the  network  side  to  the  user  side,  one 
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may  test  an  NTl  at  the  U  reference  point,  NT2  at  T,  or  TE  at  S  or  S/T.  Similarly,  for  testing  from  the 
other  direction,  one  may  test  NT2  at  S,  NTl  at  T,  or  ET  at  U.  The  tester  must  often  perform  more  than 
one  role.  For  example,  to  test  layer  2  or  layer  3  implementation  of  a  TE  at  the  S/T  reference  point,  the 
tester  must  provide  the  layer  1  signals  from  the  NTl  and  layers  2  or  3  messages  from  the  ET.  Furthermore, 
additional  test  equipment  might  be  needed  beyond  the  SUT;  for  example,  when  testing  an  NT2  from  the 
user  side  at  the  S  reference  point,  the  supports  for  the  NTl  and  ET  must  also  be  provided.  The  exact  test 
set-up  requirements  depend  on  how  much  of  the  functional  group  the  SUT  supports. 

Table  2.1.  Basic  Test  Configurations 


Configuration 

System  Under 

Reference 

Tester's  Role 

Test's  Role 

Point 

NU.l 

TE 

S 

NT2 

NU.2 

TE 

S/T 

NTl 

NU.3 

NT2 

T 

NTl 

NU.4  • 

NTl 

U 

ET 

UN.l 

NT2 

S 

TE 

UN.2 

NTl 

S/T 

TE 

UN.3 

NTl 

T 

NT2 

UN.4 

ET 

U 

NTl 

The  NTl  function  may  be  embedded  in  a  TE  or  NT2. 


Note  that  configurations  NU.l  and  UN.l,  and  configurations  NU.l  and  UN.l  each  have  the  same  setup,  but 
the  roles  of  the  tester  and  lUT  are  reversed.  Since  the  ISDN  protocols  are  not  always  symmetric  across 
the  network-user  interface,  one  can  not  share  the  same  test  script  by  simply  exchanging  roles.  Each 
configuration  requires  its  own  test  script. 

Given  a  test  configuration,  conformance  testing  is  further  divided  into  smaller  groups  by  protocol  layers  — 
the  Physical  Layer  (Layer  1)  at  the  S/T  [6,  7,  13,  16]  and  U  [12,  14]  reference  points,  the  Data  Link  Layer 
(Layer  2)  [8,  15],  and  the  Network  Layer  (Layer  3)  [9,  10,  17,  18].  In  addition,  grouping  is  necessary  by 
type  of  services  (bearer  circuit  switching,  bearer  packet  switching,  supplementary  services,  etc.),  and  type 
of  interfaces  (basic  rate  access  and  primary  rate  access). 

The  conformance  tests  described  in  this  series  of  documents  focus  on  the  basic  access  characteristics  and 
signalling  protocols  specified  in  the  ISDN  standards  Usted.  For  thoroughness,  conformance  testing  should 
also  be  done  for  the  bearer  services  covered  by  ISDN,  e.g.,  X.25  and  Group  4  FAX.  However,  tests  for 
these  services  are  beyond  the  scope  of  this  document. 

Currently,  most  of  the  ISDN  conformance  testing  activities  focus  on  testing  of  ISDN  terminal  equipment, 
since  there  are  more  terminal  devices  from  different  vendors  than  network  switching  equipment. 

3  CONFORMANCE  TEST  SUITES  FOR  ISDN 

The  ISDN  Conformance  Testing  (ICOT)  group  of  the  NIUF  has  been  developing  conformance  test  suites 
for  the  protocol  standards  for  ISDN  layers  1,  2,  and  3.  Separate  test  suites  for  the  user  and  networic  side 
equipment,  and  for  the  Basic  Rate  (BRI)  and  Primary  Rate  (PRI)  interfaces  have  been,  or  will  be. 
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developed.  It  is  the  intent  of  this  group  to  submit  these  industry  agreements  to  the  related  standard  bodies 
for  adoption  as  national  or  international  standards,  as  appropriate. 

The  following  sections  outline  the  abstract  test  suites  which  will  be  published  in  other  documents  as  part 
of  this  group  of  NIST  Special  Publications.  Included  in  this  outline  are  brief  descriptions  of  the  test 
specifications  and  test  configurations,  as  described  in  the  Scope  section  of  this  document,  for  ISDN  layers 
1,  2,  and  3,  that  are  being  developed  in  the  NIUF.  Additional  test  specifications  include  supplementary 
services,  terminal  adaptor,  and  packet  mode  bearer  services.  With  the  exception  of  the  layer  1  test  suite, 
the  methodology  used  in  the  development  of  the  following  test  suites  is  based  on  International  Organization 
for  Standardization  (ISO)  9646  [22],  and  the  test  scripts  are  written  in  the  Tree  and  Tabular  Combined 
Notation,  TTCN. 

3.1  Physical  Layer  Test  Specifications 

Layer  1  —  Physical  Layer  —  The  three  parts  of  this  specification  will  describe  the  test  specifications  for 
American  National  Standard  (ANS)  Tl.605-1989  [16],  ANS  Tl.601-1988  [14]  and  ANS  Tl.408-1990  [13]. 
This  includes  testing  of  functional,  electrical,  and  mechanical  characteristics  of  devices  including  TEl,  TA, 
and  NT's  on  the  user  side.  The  tests  may  include  verification  of  framing,  line  bit  rate,  relative  bit  positions, 
D-channel  access  control,  checking  of  jitter  and  phase,  impedance,  pulse  shape  and  amplitude,  pulse  balance, 
receiver/transmitter  performance,  power  feeding  and  consumption,  start-up  procedures,  pulse  characteristics, 
frame  structure,  M-channel  bit  functions,  etc. 

Part  1:  Basic  Rate  S/T  Interface,  User  Side  [4]  describes  a  set  of  test  specifications  which  verify 
conformance  of  TEs  and  NTs  to  the  ISDN  Physical  Layer  Basic  Access  at  the  S/T  reference  point, 
as  defined  in  ANS  Tl.605-1989  [16],  and  section  4.1.1.2  of  the  NIU-Forum  Agreements  on  ISDN 
[3].  This  test  suite  is  designed  for  test  configurations  NU.l  through  NU.5  and  UN.l  through  UN.3 
in  table  2.1. 

Part  2:  Basic  Rate  U  Interface,  User  Side  will  describe  a  set  of  test  specifications  which  verify 
equipment  implementation  conformance  to  the  ISDN  Physical  Layer  Basic  Access  at  the  U 
interface,  as  defined  in  ANS  Tl.601-1988  [14]  and  section  4.1.1.1  of  the  NIU-Forum  Agreements 
on  ISDN  [3].  This  test  suite  will  be  designed  for  test  configuration  NU.4  in  table  2.1. 

Part  3:  Primary  Rate  Interface,  User  Side  will  describe  a  set  of  test  specifications  which  verify 
equipment  implementation  conformance  to  the  ISDN  Physical  Layer  Primary  Access  at  the  S,  T, 
and  U  interface,  as  defined  in  ANS  Tl.408-1990  [13]  and  the  corresponding  NIUF  Implementation 
Agreement.  This  test  suite  will  be  designed  for  test  configurations  NU.l,  through  NU.3,  UN.l 
through  UN.3  in  table  2.1. 

3.2  Data  Link  Layer  Test  Specifications 

Layer  2  —  Data  Link  Layer  —  The  two  parts  of  this  specification  will  cover  the  protocol  states  defined 
in  the  International  Telegraph  and  Telephone  Consultative  Committee  (CCITT)  Q.921  [8]  and  ANS  T1.602 
[15]  standards.  For  each  state,  three  types  of  tests  are  specified:  valid,  inopportune,  and  invalid  tests.  The 
valid  tests  cover  the  exchange  of  valid  frames;  the  inopportune  tests  check  the  lUT's  response  to  correctly 
formatted  frames  received  in  an  unexpected  state,  and  the  invalid  tests  verify  the  lUT's  error  recovery 
procedures  upon  receiving  an  incorrectly  formatted  frame. 

Part  1:  Basic  Rate  Interface,  User  Side  will  define  the  abstract  test  specifications  to  verify 
equipment  implementation  conformance  to  the  Layer  2  of  an  ISDN  at  the  user-network  interface, 
for  the  BRI  access  arrangements,  as  defined  in  ANS  Tl.602-1989  [15]  and  section  4.1.3  of  the 
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NIU-Forutn  Agreements  on  ISDN  [3].  This  test  suite  was  designed  for  test  configurations  NU.l 
and  NU.2  in  table  2.1.  This  test  suite  is  currently  the  base  text  in  CCITT  Study  Group  XI  for 
consideration  as  an  international  reconunendation. 

Part  2:  Primary  Rate  Interface,  User  Side  will  define  the  abstract  test  specifications  to  verify 
equipment  implementation  conformance  to  the  Layer  2  of  an  ISDN  at  the  user-network  interface, 
for  the  PRI  access  arrangements,  as  defined  in  ANS  Tl  .602-1989  [15]  and  NIU-Forum  Agreements 
on  ISDN  [3].  This  test  suite  will  be  designed  for  test  configurations  NU.3  and  NU.4  in  table  2.1. 

33        Network  Layer  Test  Specifications 

Layer  3  —  Network  Access  Layer  —  The  three  specifications  involving  layer  3  will  be  based  on  CCITT 
Q.931  [9]  and  ANS  T1.607  [17]  and  cover  the  D-channel  operations  for  user  side  states  in  the  following 
functional  areas:  call  setup  from  user  and  network,  call  clearing  from  user  and  network,  and  error  recovery 
mechanisms  (invalid  information  element  and  missing  mandatory  information  element).  As  in  the  Layer 
2  test  suites,  valid,  inopportune,  and  invaUd  tests  will  be  performed.  Note  that  the  operations  of  the  B 
channels  will  not  be  covered  in  these  tests. 

Part  1:  Basic  Rate  Interface  Circuit  Switch  Call  Control,  User  Side  will  define  the  abstract  test 
specifications  to  verify  equipment  implementation  conformance  to  the  Layer  3  of  an  ISDN  BRI 
at  the  user-network  interface  for  the  Basic  Call  Control  Procedures,  as  defined  by  ANS  T1.607- 
1990  [17],  section  4.1.4.1.1  of  the  NIU-Forum  Agreements  on  ISDN  [3],  and  other  corresponding 
NIUF  Implementation  Agreements.  This  test  suite  was  designed  for  test  configurations  NU.l  and 
NU.2  in  table  2.1. 

Part  2:  Primary  Rate  Interface  Circuit  Switch  Call  Control,  User  Side  will  define  the  abstract  test 
specifications  to  verify  equipment  implementation  conformance  to  the  Layer  3  of  an  ISDN  PRI  at 
the  user-network  interface  for  the  Basic  Call  Control  Procedures,  as  defined  by  ANS  Tl. 607-1990 
[17],  section  4.1.4.1.1  of  the  NIU-Forum  Agreements  on  ISDN  [3],  and  other  corresponding  NIUF 
Implementation  Agreements.  This  test  suite  will  be  designed  for  test  configurations  NU.3  and 
NU.4  in  table  2.1. 

Part  3:  Packet  Switched  Call  Control  will  define  the  abstract  test  specifications  to  verify  equipment 
implementation  conformance  to  the  Layer  3  of  an  ISDN  at  the  user-network  interface  for  the 
Packet  Switched  Call  Control  Procedures,  as  defined  by  ANS  Tl.608-1990  [18],  section  4.1.4.2 
of  the  NIU-Forum  Agreements  on  ISDN  [3],  and  other  corresponding  NIUF  Implementation 
Agreements. 

3.4        Additional  Test  Specifications 

Supplementary  Services  will  define  the  abstract  test  specifications  to  verify  implementation  conformance 
to  the  Supplementary  Services  at  the  user-network  interface,  as  defined  in  ANS  T1.610  [19],  section 
4.1.4.11.2  of  the  NIU-Forum  Agreements  on  ISDN  [3],  and  other  appropriate  ANS  documents  and  their 
corresponding  NIUF  Implementation  Agreements. 

Packet  Mode  Bearer  Services  Control  Procedures  will  define  the  abstract  test  specifications  to  verify 
implementation  conformance  to  the  Packet  Mode  Bearer  Services  Control  Procedures. 

Terminal  Adaption  will  define  the  abstract  test  specifications  to  verify  equipment  implementation 
conformance  to  the  ISDN  Circuit-Mode  Data  Terminal  Adaption  using  Statistical  Multiplexing,  as  defined 
by  ANS  T  1.6 12- 1990  [20]  and  the  corresponding  NIUF  Implementation  Agreement. 
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4  ISDN  CONFORMANCE  TESTING 


In  order  to  ensure  that  conformance  tests  produce  correct  and  consistent  results,  it  is  essential  that  test  suites 
be  standardized.  The  costs  involved,  in  terms  of  time  and  resources,  to  develop  test  scripts,  implement 
testbeds,  and  perform  tests,  is  a  strong  incentive  to  standardize  conformance  tests.  This  section  first  presents 
the  standard  for  guidelines  in  conformance  testing,  and  briefly  describes  the  conformance  test  development 
process  including  implementors'  agreements,  abstract  test  suite  development  and  validation,  and 
supplemental  information  required  for  testing  a  vendor's  product. 

4.1        Conformance  Testing  Standards 

The  ISO  and  the  CCITT  have  developed  a  standard  for  Open  Systems  Interconnection  (OSI)  conformance 
testing.  The  resulting  standards,  ISO  9646,  "OSI  Conformance  Testing  Methodology  and  Framework"  [22] 
and  CCITT  X.290  [11]  (with  the  same  title),  cover  most  aspects  of  the  conformance  testing  process  shown 
in  figure  2.  ISO  9646  is  a  five  part  standard  consisting  of: 

Part  1    General   Concepts   —  introduces 

conformance  testing  concepts  and 

terminology; 
Part  2    Abstract  Test  Suite  Specification  — 

defines  test  methods  for  test  suite 

specification,  and  types  of  tests  which 

should  be  included  in  an  abstract  test 

suite; 

Part  3  The  Tree  and  Tabular  Combined 
Notation  (TTCN)  —  defines  a 
language  for  writing  abstract  test 
suites; 

Part  4    Test    Realization    —  specifies 

requirements   and   gives  guidance 

concerning  the  realization  of  a  means 

of  testing  lUTs;  and 
Part  5    Requirements  on  Test  Laboratories 

and  CUents  for  the  Conformance 

Assessment   Process   —  specifies 

requirements    on    both    the  test 

laboratory  and  the  client,  for  the 

conduct    of    the  conformance 

assessment  process. 

Part  2  defines  abstract  test  methods:  local, 
distributed,  coordinated  and  remote  methods. 

The  remote  single  layer  testing,  as  shown  in  figure  3,  is  the  most  conunon  method  used  for  ISDN  layer  2 
and  3  protocol  testing.  The  Implementation  Under  Test  would  be  either  the  layer  2  or  layer  3 
implementation,  which  would  exchange  layer  N  (2  or  3)  Protocol  Data  Units  (PDUs)  with  the  tester  through 
the  lower  layer  services.  The  lower  layer  services  for  the  lUT  and  lower  tester  are  assumed  to  be  available 
and  are  not  under  test.  The  "lower  tester"  refers  to  the  means  of  providing  indirect  control  and  observation 
of  the  lower  boundary  of  the  lUT  via  the  underlying  service  provider.  There  is  one  Point  of  Control  and 
Observation  (PCO)  where  the  lower  tester  observes  the  messages  being  exchanged  between  the  lower  tester 
and  the  lUT  and  assigns  verdicts  (test  results)  accordingly. 


STANDARDS 


PICS:   Protocol  ImplcmenUtion 
Conf6nnmncc  Statement 

PKIT:  Protocol  Implementatloii 


IMPLEMENTORS' 
AGREEMENTS 


ABSTRACT 
CONFORMANCE 
TEST  SUITE 


EXECUTABLE 
TEST  SUITE 

ON  TEST 
EQUIPMENT 


TESTING 
OPERATIONS 


TEST  REPORTS 


Extra  Information  for  Teiting 


Figure  2.  Conformance  Testing  Steps. 
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For  the  remote  single  layer  testing,  the 
upper  layer  support  of  the  JUT  is  not 
specified,  and  no  explicit  test  coordination 
procedures  are  assumed,  although  they 
may  be  present.  As  an  example,  during 
testing  of  the  ISDN  Layer  2  protocol,  the 
tester  may  require  the  lUT  to  send  a  link 
setup  frame,  which  would  require  some 
activities  from  Layer  3.  The  tester  may 
only  be  able  to  generate  a  request  to  the 
test  oj)erator  for  necessary  manual 
intervention  to  the  lUT  to  trigger  the  link 
setup.  The  exact  method  which  the  tester 
uses  to  accomplish  the  Unk  initiation  is 

not  defined  in  an  abstract  test  script.  It  is  outside  of  the  scope  of  the  requirements  of  this  test  method  in 
an  abstract  test  script  and  is  dependent  on  the  particular  implementation  of  the  lUT. 


N-l  Serrlce  ProTider 


Figure  3.  Remote  Single  Layer  Test  Method. 


The  TTCN  language,  defined  in  Part  3  of  ISO  9646,  allows  one  to  write  a  precise  description  of  the 
behavior  of  both  the  tester  and  the  lUT,  as  well  as  the  exact  contents  of  the  Protocol  Data  Units  exchanged. 
TTCN  has  two  forms,  the  tabular  or  graphic  form  (TTCN.GR)  for  human  reading,  and  the  machine 
processable  form  (TTCN.MP)  for  computer  processing.  Figure  4  is  a  sample  of  a  test  script  written  in 
TTCN.GR  taken  from  the  NIUF  LAPD  test  suite.  A  test  case  consists  of  a  sequence  of  frames  sent 
(indicated  by  !)  and  received  (indicated  by  ?)  by  the  tester,  and  related  timing  operations  specified  in  the 
Behavior  Description  Column.  Each  event  may  specify  the  PDU  name  and  its  exact  contents  in  the 
Behavior  Description  column  and  the  Constraints  Reference  column,  respectively.  The  PDU  and  constraint 
names  refer  to  entries  in  other  tables  in  the  test  suite. 


Test  Case  Dynamic  Behaviour 

Reference:  LAPD/MFO/S70/DL70_V14 

Identifier  DL70_V14 

Purpose:  Verify  tfiat  tfie  lUT  sends  a  SABME/P=1  in  response  to  a  DM/F=0  in 

Multiple  Frame  Establisfied  state  (7.0).  The  lUT  is  expected  to  enter  Awaiting 

Establishment  state  after  sending  the  SABME/P=1. 

Defaults  Reference: 

Behaviour  Description 

Label 

Constraint  Reference 

V 

Comments 

DL70_V14 

+DL70_PREAf^/IBLE 

\DM 

DMO_NR 

F=0 

START  Td 

?SABME 

L700 

SABN/IE1_UC 

(P) 

P=1 

+DL51_VERIFICATI0N 

+DL_POSTAMBLE 

+DL70_UNEXPECTED 

GOTO  L700 

70THERWISE 

(F) 

+DL_P0STAMBLE 

7TIMEOUT  Td 

(F) 

+DL_POSTAMBLE 

Figure  4.  Example  of  an  ISDN  Layer  2  (LAPD)  test  script  in  TTCN. 
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4.2        Conformance  Testing  Development 


Several  steps  are  required  to  develop  conformance  tests  from  the  protocol  specification.  These  steps,  shown 
in  figure  2,  include  the  development  of  an  abstract  conformance  test  suite,  the  implementation  of  executable 
test  scripts  on  test  equipment,  the  testing  operations,  and  the  preparation  of  the  test  reports.  The  following 
sections  describe  the  steps  through  which  the  abstract  test  suites  in  this  series  were  developed  and  the  initial 
steps  in  progressing  to  an  executable  test  suite.  The  details  of  the  testing  operations  and  test  reporting 
requirements  may  be  provided  in  other  publications.  The  fu-st  step  in  the  development  of  an  abstract  test 
suite  begins  with  the  precise  interpretation  of  the  base  standards. 

4.2.1  Implementors'  Agreements 

Protocol  and  interface  standards  are  specifications  for  products  and  services.  Most  standards  are  written 
in  natural  language  (e.g.,  EngUsh,  French)  and  are  subject  to  different  interpretations.  Since  a  standard  may 
involve  many  options  and  variable  fields,  the  test  suite  developer  may  need  to  rely  on  additional 
implementors'  agreements  which  specify  or  constrain  the  choice  of  options  and  allowable  fields.  North 
American  Implementors'  Agreements  for  ISDN  are  developed  by  the  Access  Signalling  Working  Group  in 
the  Implementor's  Workshop  of  the  NIUF.  Implementors'  Agreements  specify  1)  options  to  be 
implemented  (or  excluded)  and  the  PDU  codings;  2)  stacks  of  protocol  with  options  preselected,  when 
offering  specific  services;  and  3)  test  suites  to  be  used  for  assessing  conformance  of  the  items  above.  The 
Implementors'  Agreements  determine  the  set  of  test  suites  required  for  testing  conformance  of  a  stack  of 
protocols,  and  the  selection  of  test  cases  and  portions  of  the  test  scripts  within  a  test  case  for  a  given 
standard.  Once  a  set  of  implementor's  agreements  are  in  place  for  a  particular  protocol,  the  development 
of  an  abstract  test  suite  begins. 

4.2.2  Abstract  Conformance  Test  Suite  Development 

The  ISDN  Conformance  Testing  (ICOT)  Working  Group  was  formed  as  a  permanent  implementor's 
subgroup  of  the  NIUF  to  develop  abstract  conformance  test  suites  based  on  the  NIUF  Implementor's 
Agreements.  ICOT  consists  of  testing  experts  from  organizations  such  as  AT&T  Bell  Laboratories,  Bell 
Atlantic,  Bell  Northern  Research,  Bell  Communications  Research  (Bellcore),  Corporation  for  Open  Systems 
(COS),  IBM,  IDACOM/Hewlett  Packard,  NIST,  Northern  Telecom,  NYNEX,  TEKELEC,  and 
Telecommunications  Techniques  Corporation.  Test  suite  development  for  the  Layer  2  test  suite,  for 
example,  began  with  a  complete  review  of  the  protocol  standard  Q.921,  to  determine  the  desired  coverage 
by  the  test  suite.  The  test  scripts  were  developed  in  the  TTCN,  reviewed,  and  validated  by  the  participating 
members  during  numerous  NIUF  and  interim  meetings.  The  validation  process  is  described  below. 

4.2.3  Test  Suite  Validation 

The  validation  of  test  suites  was  provided  by  different  means  during  the  development  of  the  abstract  test 
suite.  First,  selected  groups  of  test  cases  were  coded  into  executable  test  cases  for  test  equipment  and 
evaluated  in  parallel  to  the  test  suite  development.  The  results  of  these  preliminary  tests  were  reported  to 
ICOT,  and  refinements  were  discussed.  The  entire  test  suite  was  coded  as  executable  tests  and  evaluated 
during  the  fmal  development  of  the  test  suite. 

A  second  validation  method  was  developed  at  NIST  during  the  development  of  the  NIUF  Layer  2  (LAPD) 
test  suite,  and  subsequently  used  for  test  suites  for  other  protocols.  Figure  5  shows  the  system  development 
process.  Based  on  the  Q.921  states/events  matrix,  a  protocol  model  of  the  LAPD  protocol  standard,  was 
developed  using  the  formal  description  language  Estelle  [21].  It  simulates  the  behavior  of  the  lUT  and  can 
be  used  to  exercise  most  of  the  test  cases  in  the  test  suite.  In  parallel,  a  tester  was  implemented  using  the 
TTCN-to-C  translator,  described  in  section  7.  The  abstract  test  suite  is  compiled  into  an  executable  test 
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suite  in  C  code.  As  part  of  this 
process,  both  semantics  and 
syntax  cliecking  is  perfonned  on 
each  test  case.  The  protocol 
model  of  the  implementation  and 
the  tester  run  concurrently  as  two 
independent  processes  on  a 
UNIX™  system.  The  Layer  1 
Service  Provider  function,  shown 
in  figure  3,  is  simulated  by  means 
of  a  UNIX™  inter-process 
communication  facility  called 
shared  memory.  In  this  system, 
an  abbreviated  Layer  3  software  is 
implemented  to  realize  the  test 
coordination  procedures  required 
by  the  lower  tester.  Thus,  no  real 
ISDN  hardware  is  required.  At 
the  end  of  each  test  case,  the  test 
log  and  the  simulator  execution 
trace  are  compared  to  find  any 
discrepancies  between  the  test 
script  and  the  behavior  of  the 
lUT. 

USE  OF  PICS  AND  PIXIT 


LAPD  Standard 
Q.921/ri.602 


NIU-Fbium 
ImpleincntoiB'  Agreements 


State/Events 
Matrix 


Test  Suite 
in  TTCN.MP 


Reference  Model 
in  ESTELLE 


TTCN  *  C  Compiler 


ESTELLE  CompUci 


LAPD  Tester 
(a  UNIX  Process) 


LAPD  Simulator 
(a  UNDC  Process) 


Physical  Layer  Interface 
(UNIX  Interprocess  Communication) 


Test  Log 


Execution  Trace 


It  is  likely  that  an  entire  test  suite  Figure  5.  LAPD  Test  Script  Validation  System, 

is  not  applicable  for  all  lUTs. 
Therefore,     a    test  selection 

procedure  must  be  performed  to  determine  the  applicability  of  each  test  case  to  a  particular  TUT.  Such 
selection  is  based  on  the  Protocol  Implementation  Conformance  Statement  (PICS)  and  the  Protocol 
Implementation  Extra  Information  for  Testing  (PKIT).  The  guidelines  for  developing  both  PICS  and  PIXIT 
are  provided  in  ISO  9646  [22]. 

The  PICS  proforma,  in  the  form  of  a  questionnaire,  becomes  the  PICS  when  completed  by  a  supplier  for 
an  ISDN  implementation.  The  PICS  specifies  the  capabilities  and  options  which  have  been  implemented, 
and  any  features  which  have  been  omitted,  so  that  the  implementation  can  be  tested  for  conformance  against 
relevant  requirements,  and  against  those  requirements  only.  Static  conformance  assessment  is  based  upon 
information  provided  about  the  lUT  in  the  PICS;  e.g.,  are  all  of  the  mandatory  features  implemented,  and 
are  related  conditional  options  implemented  when  there  are  inter-dependencies.  The  conformance  testing 
guidelines  of  ISO  9646  recommends  that  the  PICS  proforma  become  part  of  the  protocol  standard. 

Additional  information  related  to  the  lUT  and  its  environment  is  needed  before  a  test  campaign  can 
commence;  this  information  is  contained  in  a  PIXIT  and  supplements  information  contained  in  a  PICS.  The 
PIXIT  questionnaire  should  be  completed  by  the  manufacturer  of  the  lUT,  or  the  test  laboratory  client,  when 
submitting  an  lUT  for  testing.  The  PIXIT  contains  information  regarding  system  specific  details  required 
by  the  test  laboratory  to  run  the  test  suite.  Timer  values,  for  example,  may  be  specified  in  the  PICS  as  a 
range  of  values  for  a  particular  timer.  The  PIXIT  questionnaire  may  request  a  specific  timer  value.  In  this 
way,  the  PIXIT  provides  a  refinement  of  the  PICS  information.  The  values  of  test  variables  and  parameters 
for  conditions  regarding  test  case  selection  are  derived  by  the  test  laboratory  from  the  information  supplied 
in  the  PIXIT.  ISO  9646  recommends  that  the  PIXIT  questionnaire  be  part  of  the  related  abstract  test  suite. 


6  CONFORMANCE  VS.  INTEROPERABILITY  TESTING 


Since  confonnance  testing  is  done  in  a  controlled  environment,  the  fact  that  equipment  passes  conformance 
tests  does  not  guarantee  that  it  will  interoperate  without  problems.  Interoperability  may  depend  on  1)  all 
manufacturers  implementing  the  same  set  of  options;  2)  system  loads,  which  may  impact  the  availabiUty 
buffers;  and  3)  implementation  and  synchronization  of  timers,  i.e.,  a  mismatched  timer  may  cause  a  call  to 
be  aborted.  Conformance  testing  only  increases  the  confidence  that  the  equipment  was  implemented  as 
specified  in  the  standard.  External  factors,  such  as  the  condition  of  the  communication  port,  may  also  affect 
the  outcomes  of  the  confonnance  tests.  Furthermore,  conformance  testing  does  not  imply  complete  and 
exhaustive  testing  of  every  aspect  of  a  standard.  The  dynamic  behaviors  of  ISDN  protocols  are  so  complex 
that  it  is  not  economically  feasible  to  perform  exhaustive  testing.  Conformance  testing  is  the  first  step  in 
product  quality  testing  and  is  supplemented  by  interoperability  tests  and  system  load  tests. 

7  THE  NIST  TTCN  TRANSLATOR 

The  purpose  of  developing  the  TTCN-to-C  language  translator  was  two-fold:  1)  to  provide  a  tool  to 
industry  which  would  expedite  the  process  of  converting  an  abstract  test  suite  (specified  in  TTCN)  into  an 
executable  test  suite  (specified  in  the  C  language),  and  2)  to  provide  a  tool  which  could  be  used  in  the  test 
suite  validation  process  described  above. 

The  NIST  TTCN  translator  [2]  accepts  a  test  script  in  machine  processable  form,  TTCN.MP,  and  translates 
it  into  C  language  statements.  It  can  be  used  to  ttansform  an  abstract  test  suite  into  an  executable  test 
system  with  minimum  effort.  Figure  6  shows  the  process  needed  to  implement  an  executable  test  system 


Protocol  dependent 
support  programs 


Abstract 

test 

suite 


TTCN 


Editor 


Test  suite 
in  TrCN.MP 


TTCN 


Translator 


Test  suite  in 
C  source 
programs 


CompUe  & 


Link 


Testbed  machine 
dependent  support 
programs 


Real 

test 

system 


Figure  6.  Implementation  of  a  Test  System  Using  the  NIST  TTCN  Translator. 


using  the  NIST  translator.  The  abstract  test  suite  in  TTCN.MP  form  is  generated  using  one  of  the  TTCN 
tools  described  above.  It  is  then  input  into  the  TTCN  translator  which  produces  a  set  of  C  language 
programs.  These  programs  are  compiled  on  the  target  testbed  and  bound  with  other  support  programs  to 
make  an  executable  test  system.  The  protocol  and  system  dependent  support  programs  include  such  items 
as  formatting  and  analyzing  PDUs,  controlling  the  ISDN  ports,  and  handling  timers  or  clocks.  The 
translator  has  been  used  to  validate  the  correcmess  of  the  NIUF  LAPD  test  suite,  and  to  implement  real  test 
systems. 
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ANNEX  A  ABBREVIATIONS 


ANS  American  National  Standard 

ASP  Abstract  Service  Primitives 

BRI  Basic  Rate  Interface 

CCITT  International  Telegraph  and  Telephone  Consultative  Committee 

ET  Exchange  Termination 

FIPS  Federal  Information  Processing  Standard 

ICOT  ISDN  Confonnance  Testing  Working  Group  of  NIUF 

ISDN  Integrated  Services  Digital  Network 

ISO  International  Organization  for  Standardization 

lUT  Implementation  Under  Test 

LAPD  Link  Access  Procedure  on  the  D-channel 

NIST  National  Institute  of  Standards  and  Technology 

NIUF  North  American  ISDN  Users'  Forum 

NT  Network  Termination  (NTl  and  NT2) 

OSI  Open  Systems  Interconnection 

PCO  Point  of  Control  and  Observation 

PDU  Protocol  Data  Unit 

PICS  Protocol  Implementation  Conformance  Statement 

PIXIT  Protocol  Implementation  Extra  Information  for  Testing 

PRI  Primary  Rate  Interface 

TA  Terminal  Ad^tor 

TEl  Terminal  Equipment  of  type  1  (ISDN) 

TE2  Terminal  Equipment  of  type  2  (Non-ISDN) 

TTCN  Tree  and  Tabular  Combined  Notation 
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NIST. 


Technical  Publications 

Periodical 

Journal  of  Research  of  the  National  Institute  of  Standards  and  Technology— Reix)rts  NIST 
research  and  development  in  those  discipHnes  of  the  physical  and  engineering  sciences  in  which 
the  Institute  is  active.  These  include  physics,  chemistiy,  engineering,  mathematics,  and  computer 
sciences. 

Papers  cover  a  broad  range  of  subjects,  with  major  emphasis  on  measurement  methodology  and 
the  basic  technology  underlying  standardization.  Also  included  from  time  to  time  are  survey 
articles  on  topics  closely  related  to  the  Institute's  technical  and  scientific  programs.  Issued  six 
times  a  year. 


Nonperiodicals 


Monographs— Major  contributions  to  the  technical  literature  on  various  subjects  related  to  the 
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Handbooks  — Recommended  codes  of  engineering  and  industrial  practice  (including  safeW  codes) 
developed  in  cooperation  with  interestecT  industries,  professional  organizations,  and  regulatory 
bodies. 

Special  Publications  — Include  proceedings  of  conferences  sponsored  by  NIST,  NIST  annual 
reports,  and  other  special  publications  appropriate  to  this  grouping  such  as  wall  charts,  pocket 
cards,  and  bibliographies. 

Applied  Mathematics  Series— Mathematical  tables,  manuals,  and  studies  of  special  interest  to 
physicists,  engineers,  chemists,  biologists,  mathematicians,  computer  programmers,  and  others 
engaged  in  scientific  and  technical  work. 

National  Standard  Reference  Data  Series— Provides  quantitative  data  on  the  physical  and  chemical 
properties  of  materials,  compiled  from  the  world's  literature  and  critically  evaluated.  Developed 
unaer  a  worldwide  program  coordinated  by  NIST  under  the  authority  of  the  National  Standard 
Data  Act  (Public  Law  90-396).  NOTE:  The  Journal  of  Physical  and  Chemical  Reference  Data 
(JPCRD)  is  published  bimonthly  for  NIST  by  the  American  Chemical  Society  (ACS)  and  the 
American  Institute  of  Physics  (AIP).  Subscriptions,  reprints,  and  supplements  are  available  from 
ACS,  1155  Sixteenth  St.,  NW.,  Washington,  DC  20056. 

Building  Science  Series— Disseminates  technical  information  developed  at  the  Institute  on  building 
materials,  components,  systems,  and  whole  structures.  The  series  presents  research  results,  test 
methods,  and  performance  criteria  related  to  the  structural  and  environmental  functions  and  the 
durability  and  safety  characteristics  of  building  elements  and  systems. 

Technical  Notes  —  Studies  or  reports  which  are  complete  in  themselves  but  restrictive  in  their 
treatment  of  a  subject.  Analogous  to  monographs  but  not  so  comprehensive  in  scope  or  definitive 
in  treatment  of  the  subject  area.  Often  serve  as  a  vehicle  for  final  reports  of  work  performed  at 
NIST  under  the  sponsorship  of  other  government  agencies. 

Voluntary  Product  Standards  —  Developed  under  procedures  published  by  the  Department  of 
Commerce  in  Part  10,  Title  15,  of  the  Code  of  Federal  Regulations.  The  standards  establish 
nationally  recognized  requirements  for  products,  and  provide  all  concerned  interests  with  a  basis 
for  common  understanding  of  the  characteristics  of  the  products.  NIST  administers  this  program 
as  a  supplement  to  the  activities  of  the  private  sector  standardizing  organizations. 
Consumer  Information  Series  —  Practical  information,  based  on  NIST  research  and  experience, 
covering  areas  of  interest  to  the  consumer.  Easily  understandable  language  and  illustrations 
provide  useful  background  knowledge  for  shopping  in  today's  technological  marketplace. 
Order  the  above  NIST  publications  from:  Superintendent  of  Documents,  Government  Printing  Office, 
Washington,  DC  20402. 

Order  the  following  NIST  publications— FIPS  and  NISTIRs—from  the  National  Technical  Information 
Service,  Springfield,  VA  22161. 

Federal  Information  Processing  Standards  Publications  (FIPS  PUB)  —  Publications  in  this  series 
collective^  constitute  the  Federal  Information  Processing  Standards  Register.  The  Register  serves 
as  the  official  source  of  information  in  the  Federal  Government  regarding  standards  issued  by 
NIST  pursuant  to  the  Federal  Property  and  Administrative  Services  Act  of  1949  as  amended, 
Public  Law  89-306  (79  Stat.  1127),  and  as  implemented  by  Executive  Order  11717  (38  FR  12315, 
dated  May  11,  1973)  and  Part  6  of  Title  15  CFR  (Code  of  Federal  Regulations). 
NIST  Interagency  Reports  (NISTIR)— A  special  series  of  interim  or  final  reports  on  work 
performed  by  NIST  for  outside  sponsors  (ooth  government  and  non-government).  In  general, 
initial  distribution  is  handled  by  the  sponsor;  public  distribution  is  by  the  National  Technical 
Information  Service,  Springfield,  VA  22161,  in  paper  copy  or  microfiche  form. 
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